Methods and Systems for Determining Abnormal Cardiac Activity

ABSTRACT

The systems and methods can accurately and efficiently determine abnormal cardiac activity from motion data and/or cardiac data using techniques that can be used for long-term monitoring of a patient. In some embodiments, the method for using machine learning to determine abnormal cardiac activity may include receiving one or more may include applying a trained deep learning architecture to each tensor of the one or more periods of time to classify each window and/or each period into one or more classes using at least the one or more signal quality indices for the cardiac data and the motion data and cardiovascular features. The deep learning architecture may include a convolutional neural network, a bidirectional recurrent neural network, and an attention network. The one or more classes may include abnormal cardiac activity and normal cardiac activity.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/437,457 filed Dec. 21, 2016. The entirety of this application ishereby incorporated by reference for all purposes.

BACKGROUND

Arrhythmias, which are characterized by abnormal heart rates, can causefatal conditions, such as strokes or sudden cardiac death, as well as bean indicator of a serious condition, such as heart disease. One of themost common form arrhythmia is atrial fibrillation (Afib).

Generally, arrhythmias are detected from continuous ECG(electrocardiographic) monitoring using ECG devices that are usedperiodically over a few weeks. These monitoring techniques can requirethe use of multiple electrodes, for example, patches and implantabledevices, making them cumbersome and sometimes invasive for the user.These techniques can also be costly, although being used for short-term.Further, many patients or subjects suffering from atrial fibrillation(Afib) can be asymptomatic and the ECG monitoring may not detect unknownAfib. Thus, current ECG methods and devices can also be inefficient indetecting Afib.

There has been some developments in using wearable devices that detectphotoplethysmogram (PPG) data. However, PPG recordings can be noisy dueto movement of the user and the noisy recordings can mask occurrences ofAfib.

SUMMARY

Thus, there is need for systems and methods that provide a costeffective, accurate detection of abnormal cardiac activity and that canbe used for long-term monitoring.

The disclosure relates to systems and methods that can accuratelydetermine abnormal cardiac activity of a subject using a deep learningarchitecture.

In some embodiments, the methods may include computer-implemented methodfor using machine learning to determine abnormal cardiac activity of asubject. The method may include receiving one or more periods of time ofcardiac data and motion data for a subject. Each period of timeincluding more than one window of the cardiac data and the motion data.The method may further include determining one or more signal qualityindices for each window of the cardiac data and the motion data of theone or more periods of time. The method may also include extracting oneor more cardiovascular features for each period of time using at leastthe cardiac data, the motion data, and the one or more signal qualityindices for the cardiac data and the motion data. The method may includeapplying a tensor transform to the cardiac data and/or the motion datato generate a tensor for each window of the one or more periods of time.The method may also include applying a trained deep learningarchitecture to each tensor of the one or more periods of time toclassify each window and/or each period into one or more classes usingat least the one or more signal quality indices for the cardiac data andthe motion data and cardiovascular features. In some embodiments, thedeep learning architecture may include a convolutional neural network, abidirectional recurrent neural network, and an attention network. Theone or more classes may include abnormal cardiac activity and normalcardiac activity. The method may further include generating a reportincluding a classification of cardiac activity of the subject for theone or more periods based on the one or more classes.

In some embodiments, the method may further include receiving subjectcontextual information for the subject. The subject contextualinformation may include medical history and demographic information. Theextracting may use one or more subject information features related tothe subject contextual information to extract one or more cardiovascularfeatures for each period of time, and the trained deep learningarchitecture may use the one or more subject information features toclassify the cardiac activity for each window of the period.

In some embodiments, the tensor transform may be applied to the cardiacdata and the motion data for each window.

In some embodiments, the method may further include determining aquality channel for each window based on the one or more signal qualityindices for the cardiac data and the motion data. The quality channelmay correspond to a channel in each window having the one more qualityindices that is higher than remaining channels in each channel.

In some embodiments, the applying the deep learning architecture mayinclude encoding each tensor for each window of the one or more periodsusing the deep convolutional network into one or more deep learningfeatures associated with cardiac activity. The applying may also includeapplying the bidirectional recurrent network to determine a probabilitythat each window of the one or more periods belongs to a class of theone or more classes. The bidirectional recurrent network may use the oneor more deep learning features, the one more signal quality indices forthe cardiac data and/or motion data, and/or one or more cardiovascularfeatures to classify each window of the one or more periods. Theapplying may also include determining the classification of cardiacactivity for each window of the one or more periods and/or each periodby applying the attention network to the probability for each window ofthe one or more periods.

In some embodiments, the attention network may determine a score foreach window and/or each period, the score representing theclassification of cardiac activity. In some embodiments, when theclassification of cardiac activity includes abnormal cardiac activity, awindow of each period having a highest score may represent the windowincluding the abnormal cardiac activity.

In some embodiments, the computer readable media may include anon-transitory computer-readable storage medium storing instructions forusing machine learning to determine abnormal cardiac activity of asubject. The instructions may include receiving one or more periods oftime of cardiac data and motion data for a subject. Each period of timeincluding more than one window of the cardiac data and the motion data.The instructions may further include determining one or more signalquality indices for each window of the cardiac data and the motion dataof the one or more periods of time. The instructions may also includeextracting one or more cardiovascular features for each period of timeusing at least the cardiac data, the motion data, and the one or moresignal quality indices for the cardiac data and the motion data. Theinstructions may include applying a tensor transform to the cardiac dataand/or the motion data to generate a tensor for each window of the oneor more periods of time. The instructions may also include applying atrained deep learning architecture to each tensor of the one or moreperiods of time to classify each window and/or each period into one ormore classes using at least the one or more signal quality indices forthe cardiac data and the motion data and cardiovascular features. Insome embodiments, the deep learning architecture may include aconvolutional neural network, a bidirectional recurrent neural network,and an attention network. The one or more classes may include abnormalcardiac activity and normal cardiac activity. The instructions mayfurther include generating a report including a classification ofcardiac activity of the subject for the one or more periods based on theone or more classes.

In some embodiments, the instructions may further include receivingsubject contextual information for the subject. The subject contextualinformation may include medical history and demographic information. Theextracting may use one or more subject information features related tothe subject contextual information to extract one or more cardiovascularfeatures for each period of time, and the trained deep learningarchitecture may use the one or more subject information features toclassify the cardiac activity for each window of the period.

In some embodiments, the tensor transform may be applied to the cardiacdata and the motion data for each window.

In some embodiments, the instructions may further include determining aquality channel for each window based on the one or more signal qualityindices for the cardiac data and the motion data. The quality channelmay correspond to a channel in each window having the one more qualityindices that is higher than remaining channels in each channel.

In some embodiments, the applying the deep learning architecture mayinclude encoding each tensor for each window of the one or more periodsusing the deep convolutional network into one or more deep learningfeatures associated with cardiac activity. The applying may also includeapplying the bidirectional recurrent network to determine a probabilitythat each window of the one or more periods belongs to a class of theone or more classes. The bidirectional recurrent network may use the oneor more deep learning features, the one more signal quality indices forthe cardiac data and/or motion data, and/or one or more cardiovascularfeatures to classify each window of the one or more periods. Theapplying may also include determining the classification of cardiacactivity for each window of the one or more periods and/or each periodby applying the attention network to the probability for each window ofthe one or more periods.

In some embodiments, the attention network may determine a score foreach window and/or each period, the score representing theclassification of cardiac activity. In some embodiments, when theclassification of cardiac activity includes abnormal cardiac activity, awindow of each period having a highest score may represent the windowincluding the abnormal cardiac activity.

In some embodiments, the systems may include a system for using machinelearning to determine abnormal cardiac activity of a subject. The systemmay include a memory; and one or more processors. In some embodiments,the one or more processors may be configured to cause receiving one ormore periods of time of cardiac data and motion data for a subject. Eachperiod of time including more than one window of the cardiac data andthe motion data. The one or more processors may further be configured tocause determining one or more signal quality indices for each window ofthe cardiac data and the motion data of the one or more periods of time.The one or more processors may also be configured to cause extractingone or more cardiovascular features for each period of time using atleast the cardiac data, the motion data, and the one or more signalquality indices for the cardiac data and the motion data. The one ormore processors may be configured to cause applying a tensor transformto the cardiac data and/or the motion data to generate a tensor for eachwindow of the one or more periods of time. The one or more processorsmay be configured to cause applying a trained deep learning architectureto each tensor of the one or more periods of time to classify eachwindow and/or each period into one or more classes using at least theone or more signal quality indices for the cardiac data and the motiondata and cardiovascular features. In some embodiments, the deep learningarchitecture may include a convolutional neural network, a bidirectionalrecurrent neural network, and an attention network. The one or moreclasses may include abnormal cardiac activity and normal cardiacactivity. The one or more processors may be configured to causegenerating a report including a classification of cardiac activity ofthe subject for the one or more periods based on the one or moreclasses.

In some embodiments, the one or more processors may be furtherconfigured to cause receiving subject contextual information for thesubject. The subject contextual information may include medical historyand demographic information. The extracting may use one or more subjectinformation features related to the subject contextual information toextract one or more cardiovascular features for each period of time, andthe trained deep learning architecture may use the one or more subjectinformation features to classify the cardiac activity for each window ofthe period.

In some embodiments, the tensor transform may be applied to the cardiacdata and the motion data for each window.

In some embodiments, the one or more processors may be configured tocause determining a quality channel for each window based on the one ormore signal quality indices for the cardiac data and the motion data.The quality channel may correspond to a channel in each window havingthe one more quality indices that is higher than remaining channels ineach channel.

In some embodiments, the applying the deep learning architecture mayinclude encoding each tensor for each window of the one or more periodsusing the deep convolutional network into one or more deep learningfeatures associated with cardiac activity. The applying may also includeapplying the bidirectional recurrent network to determine a probabilitythat each window of the one or more periods belongs to a class of theone or more classes. The bidirectional recurrent network may use the oneor more deep learning features, the one more signal quality indices forthe cardiac data and/or motion data, and/or one or more cardiovascularfeatures to classify each window of the one or more periods. Theapplying may also include determining the classification of cardiacactivity for each window of the one or more periods and/or each periodby applying the attention network to the probability for each window ofthe one or more periods.

In some embodiments, the attention network may determine a score foreach window and/or each period, the score representing theclassification of cardiac activity. In some embodiments, when theclassification of cardiac activity includes abnormal cardiac activity, awindow of each period having a highest score may represent the windowincluding the abnormal cardiac activity.

In some embodiments, the cardiac data may include ECG and/or PPG data.In some embodiments, the motion data may include accelerometer data.

In some embodiments, the cardiac data and/or the motion data may bereceived from one or more sensor data collections device including oneor more cardiac sensors configured to detect cardiac data and one ormore motion sensors configured to detect motion data. In someembodiments, the one or more sensor data collection devices includes awearable device, such as a smart watch.

Additional advantages of the disclosure will be set forth in part in thedescription which follows, and in part will be obvious from thedescription, or may be learned by practice of the disclosure. Theadvantages of the disclosure will be realized and attained by means ofthe elements and combinations particularly pointed out in the appendedclaims. It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary andexplanatory only and are not restrictive of the disclosure, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure can be better understood with the reference to thefollowing drawings and description. The components in the figures arenot necessarily to scale, emphasis being placed upon illustrating theprinciples of the disclosure.

FIG. 1 shows an example of a system that can be used to determinecardiac activity according to embodiments;

FIG. 2 shows a method of determining cardiac activity according toembodiments;

FIG. 3 shows a method of classifying the cardiac activity according toembodiments;

FIG. 4 shows an example of a tensor transformation according toembodiments;

FIG. 5 shows an example of deep convolutional neural network accordingto embodiments;

FIG. 6 shows an example of a bidirectional recurrent neural networkaccording to embodiments;

FIG. 7 shows an example of an attention network according toembodiments; and

FIG. 8 shows a block diagram illustrating an example of a computingsystem.

DESCRIPTION OF THE EMBODIMENTS

In the following description, numerous specific details are set forthsuch as examples of specific components, devices, methods, etc., inorder to provide a thorough understanding of embodiments of thedisclosure. It will be apparent, however, to one skilled in the art thatthese specific details need not be employed to practice embodiments ofthe disclosure. In other instances, well-known materials or methods havenot been described in detail in order to avoid unnecessarily obscuringembodiments of the disclosure. While the disclosure is susceptible tovarious modifications and alternative forms, specific embodimentsthereof are shown by way of example in the drawings and will herein bedescribed in detail. It should be understood, however, that there is nointent to limit the disclosure to the particular forms disclosed, but onthe contrary, the disclosure is to cover all modifications, equivalents,and alternatives falling within the spirit and scope of the disclosure.

The systems and methods of the disclosure can accurately determineabnormal cardiac activity a subject (e.g., a human subject, a patient,an animal, (e.g., equine, canine, porcine, bovine, etc.), etc.). Thesystems and methods of the disclosure use more than one neural networkto determine abnormal cardiac activity based on cardiac and motion data,such as ECG or PPG data and accelerometer data. This can result inimproved performance compared to cardiac data (e.g., PPG or ECG) basedapproaches that rely only on beat detection.

As used herein, “cardiac activity” may relate to the function of theheart for a period of time. In some embodiments, the cardiac activitymay relate to normal cardiac activity (normal sinus rhythm) or abnormalcardiac activity. Abnormal cardiac activity may relate to any cardiacabnormality that can be identifiable on cardiac and/or motion data. Theabnormal cardiac activity may be arrhythmic and/or non-arrhythmic. Byway of example, an arrhythmia refers to a cardiac arrhythmia, (alsoknown as cardiac dysrhythmia), which can refer to an irregular timing ormorphology in a heart beat or sequence of beats. For example, abnormalcardiac activity may include but is not limited to arrhythmias, such asatrial fibrillation, ventricular tachycardia, sinus tachycardia, sinusbradycardia, atrial flutter, atrial tachycardia, junctional tachycardia,premature ventricular complex, premature atrial complex, ventricularpremature contraction, among others; other abnormal cardiac activity,such as acute myocardial infarction, myocardial infarction, ischemia,among others; or any combination thereof. Although the application isdescribed with respect to atrial fibrillation, the methods and systemscan be configured to detect additional and/or other abnormal cardiacactivity (e.g., arrhythmias).

FIG. 1 shows a system 100 that can determine cardiac activity usingcardiac and motion data according to embodiments. In some embodiments,the system 100 may include one or more sensor collection devices 110configured to collect at least the motion data and the cardiac data, anda cardiac activity processing device 130 configured to determine cardiacactivity using at least the motion data and the cardiac data. In someembodiments, the one or more sensor collection devices 110 may includeone or more cardiac sensors 112 and one or more motion sensors 114.

In some embodiments, the cardiac data may relate to a signal related toa function of a subject's heart. By way of example, the cardiac data mayinclude but is not limited to PPG data, ECG data, electromyographicdata, electroencephalographic data, phonocardiographic (PCG) data,ballistocaridographic data, blood pressure data, among others, or anycombination thereof. The one or more cardiac sensors 112 may include butare not limited to PPG sensor(s), ECG sensor(s), electromyographicsensor(s), electroencephalographic sensor(s), phonocardiographic (PCG)sensor(s), acoustic sensor(s), optical sensor(s), ballistocaridographicsensor(s), video or camera sensor(s), off-body sensor(s) (e.g., radarsensor(s), video or camera sensors (s)), among others, or a combinationthereof. By way of example, the one or more sensors electrocardiograph(ECG) sensors may include direct contact electrodes on the skin orcapacitive contact; opto-electrical photoplethysmography (PPG)measurements may include light source, e.g., a light emitting diode(LED) and photodetector (e.g. transistor/diode or a photodiode (PD)) asa receiver against the skin, LED and Photo diode arrays astransmitter-receiver pairs against the skin, a camera as a detector; aPCG sensors may include a Giant-Magneto-Resistance (GMR) sensors;acoustic sensors may include an acoustic sensor based microphone; andoff-body sensors may include off-body devices such as radar, cameras,LIDAR, etc.

In some embodiments, the motion data may relate to body motion of thesubject. In some embodiments, the one or more motion sensors 114 mayinclude but are not limited to an accelerometer, gyroscope, amongothers, or a combination thereof. By way of example, the accelerometermay be configured to detect accelerations of body parts of the subjectand be configured to detect motion (e.g., posture changes) of thesubject by determining changes in average orientation of theaccelerometer with respect to gravity.

In some embodiments, the cardiac sensor(s) 112 and the motion sensor(s)114 may be embedded within or otherwise coupled to (or interoperatewith) one or more sensor collection devices 110 that can be removablyattached to a user. By way of example, one sensor collection device maybe a wearable device, such as a smart watch, glasses, a headband,helmet, a smart phone attached using an attachment device (e.g., armband).

In some embodiments, the cardiac sensor(s) 110 and the motion sensor(s)120 may be embedded within or otherwise coupled to one sensor collectiondevice 110. For example, the one sensor collection device 110 may be asmart watch including at least the cardiac and motion sensors that canbe attached to an individual's wrist, for example, using a wrist band.

In some embodiments, each of the cardiac sensor(s) 110 and the motionsensor(s) 120 may be disposed within or otherwise coupled to a sensorcollection device 110 so that they are each disposed on their respectivesensor collection device. By way of example, the one or more sensorcollection devices 110, for example, for the cardiac sensor(s) 110, canbe removably attached to an individual using a patch (e.g., adhesivepatch, sticker, etc.)

In some embodiments, the one or more sensor data collection devices 110may also include one or more other sensors 116. In some embodiments, theone or more other sensors 116 may include but are not limited to athermometer, location (such as GPS), galvanic skinresponse/electrodermal activity sensors, among others, or a combinationthereof.

In some embodiments, the system 100 may further include one or moresubject information collection devices 140. The subject informationcollection device(s) 140 may include one or more devices or systems orotherwise be configured to communicate with systems or devices that areconfigured to collect and/or store the subject information. By way ofexample, the subject information (also referred to as “subjectcontextual information”) may include contextual information about thesubject, such as medical history information (e.g., history of heartdisease, current and past medication history (e.g., medication, dosages,etc.), treatment history, devices, weight, height, etc.), demographicinformation (e.g., age, gender, etc.), activity information, othercontextual information/covariates, or any combination thereof. Forexample, the subject information collection device(s) 140 may beconfigured to communicate with one or more electronic medical record(EMR) systems that store health and/or demographic information of thesubject to retrieve the medical history information. In another example,the subject information may be provided by the subject and/or anotheruser (e.g., clinician) using an interface. For example, the subject orclinician may provide the information using a mobile or computerapplication. In some embodiments, the subject information may becollected by questionnaires on psychoscial activity (e.g. PHQ9),pre-existing prior information, such as the NYHA classification. In someembodiments, the subject information collection device 140 may beconfigured to communicate with one or more applications to retrievesubject contextual information (e.g., such as fitness application(s)) toretrieve information related to fitness or physical activity).

In some embodiments, the cardiac activity determination device 130 maybe configured to determine cardiac activity based on at least the motiondata and the cardiac data (and optionally the subject information) usinga deep learning architecture. In some embodiments, the deep learningarchitecture may include more one or more (trained) deep neuralnetworks. In some embodiments, the one or more trained deep neuralnetworks may include a convolutional neural network, a bidirectionalrecurrent neural network, an attention network, among others, or acombination thereof. The one or more deep learning networks may betrained based on training samples of motion data and/or cardiac datahaving known cardiac activity features, such as known abnormal cardiacactivity, known subject information (e.g., medical and/or demographicinformation (e.g., age, medication use, device use, etc.)), amongothers, or combination thereof.

The cardiac activity determination device 130 may be configured formulti-class classification of cardiac activity using at least the motiondata and the cardiac data. For example, the cardiac activitydetermination device 130 may be configured to determine whether thecardiac activity for a period of time corresponds to the one of thefollowing classes: normal, noise, abnormal, among others, or combinationthereof. In some embodiments, abnormal class may any abnormal cardiacactivity. In some embodiments, the abnormal class may refer to one typeof abnormal activity (e.g., atrial fibrillation). In some embodiments,the abnormal class may refer to more than one type of abnormal activity(one or more arrhythmic abnormalities and/or non-arrhythmicabnormalities).

In some embodiments, the cardiac activity determination device 130 maybe embedded in or interoperate with various computing devices, such as amobile phone, a cellular phone, a smart phone, a personal computer (PC),a laptop, a notebook, a netbook, a tablet personal computer (tablet), awearable computer (e.g., smart watch, glasses etc.), among others, or acombination thereof.

In some embodiments, the one or more sensor data collection devices 110,the cardiac activity processing device 130, and/or the subjectinformation collection device 140 may be disposed within the same deviceor otherwise have connectivity via a communication network. By way ofexample, the communication network of system 100 can include one or morenetworks such as a data network, a wireless network, a telephonynetwork, or any combination thereof. The data network may be any localarea network (LAN), metropolitan area network (MAN), wide area network(WAN), a public data network (e.g., the Internet), short range wirelessnetwork, or any other suitable packet-switched network, such as acommercially owned, proprietary packet-switched network, e.g., aproprietary cable or fiber-optic network, and the like, NFC/RFID, RFmemory tags, touch-distance radios, or any combination thereof. Inaddition, the wireless network may be, for example, a cellular networkand may employ various technologies including enhanced data rates forglobal evolution (EDGE), general packet radio service (GPRS), globalsystem for mobile communications (GSM), Internet protocol multimediasubsystem (IMS), universal mobile telecommunications system (UMTS),etc., as well as any other suitable wireless medium, e.g., worldwideinteroperability for microwave access (WiMAX), Long Term Evolution (LTE)networks, code division multiple access (CDMA), wideband code divisionmultiple access (WCDMA), wireless fidelity (WiFi), wireless LAN (WLAN),Bluetooth®, Internet Protocol (IP) data casting, satellite, mobilead-hoc network (MANET), and the like, or any combination thereof.

Although the systems/devices of the system 100 are shown as beingdirectly connected, the systems/devices may be indirectly connected toone or more of the other systems/devices of the system 100. In someembodiments, a system/device may be only directly connected to one ormore of the other systems/devices of the system 100.

It is also to be understood that the system 100 may omit any of thedevices illustrated and/or may include additional systems and/or devicesnot shown. It is also to be understood that more than one device and/orsystem may be part of the system 100 although one of each device and/orsystem is illustrated in the system 100. It is further to be understoodthat each of the plurality of devices and/or systems may be different ormay be the same. For example, one or more of the devices of the devicesmay be hosted at any of the other devices.

In some embodiments, any of the devices of the system 100, for example,the cardiac activity processing device 130, may include a non-transitorycomputer-readable medium storing program instructions thereon that isoperable on a user device. A user device may be any type of mobileterminal, fixed terminal, or portable terminal including a mobilehandset, station, unit, device, multimedia computer, multimedia tablet,Internet node, communicator, desktop computer, laptop computer, notebookcomputer, netbook computer, tablet computer, personal communicationsystem (PCS) device, wearable computer (e.g., smart watch), or anycombination thereof, including the accessories and peripherals of thesedevices, or any combination thereof. FIG. 8 shows an example of a userdevice.

FIGS. 2 and 3 show methods of determining abnormal cardiac activityaccording to embodiments. Unless stated otherwise as apparent from thefollowing discussion, it will be appreciated that terms such as“encoding,” “generating,” “determining,” “displaying,” “obtaining,”“applying,” “processing,” “computing,” “selecting,” “receiving,”“detecting,” “classifying,” “calculating,” “quantifying,” “outputting,”“acquiring,” “analyzing,” “retrieving,” “inputting,” “assessing,”“performing,” or the like may refer to the actions and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices. The system forcarrying out the embodiments of the methods disclosed herein is notlimited to the systems shown in FIGS. 1 and 8. Other systems may also beused.

The methods of the disclosure are not limited to the steps describedherein. The steps may be individually modified or omitted, as well asadditional steps may be added. It will be also understood that at leastsome of the steps may be performed in parallel.

FIG. 2 illustrates a method 200 for determining cardiac activity basedon at least motion and cardiac data. For example, the method 200 mayresult in a diagnosis of abnormal cardiac activity (e.g., atrialfibrillation) or a burden thereof.

In some embodiments, the method 200 may include a step 210 of receivingthe data for the subject. In some embodiments, the step 210 may includereceiving the cardiac data 212 and the motion data 214 from the one ormore sensors (e.g., sensor(s) 112 and sensor(s) 114, respectively, forthe one or more period of times (referred to also as “time period”) viathe sensor collection device(s) 110.

In some embodiments, the step 210 may optionally include receiving thesubject information 216, for example, from the subject informationcollection device(s) 140. In some embodiments, for example, if multipleperiods of cardiac and motion data are processed for the subject, thesystem 100 may store the subject information (e.g., the featuresdetermined step 240) received from the subject information collectiondevice(s) 140.

In some embodiments, the method 200 may include a step 220 ofpre-processing the motion data and the cardiac data for each period oftime to prepare the data for classification.

In some embodiments, the pre-processing 220 may include dividing eachtime period of the motion and cardiac data into a plurality ofnon-overlapping windows of time. In some embodiments, the windows may beof any interval. For example, each window of data may be thirty seconds,less than thirty seconds (e.g., 20 seconds), more than thirty seconds(e.g., 1 minute, 2 minutes), etc. In some embodiments, each window maybe of the same size.

In some embodiments, the cardiac activity processing device 130 mayreceive a set of motion and cardiac data for a length of time (e.g., anhour period) and may divide the set into a plurality of time periods forprocessing. For example, the step 220 may include separating the hour ofmotion and cardiac data into ten-minute periods and may include furtherseparate each ten-minute period into thirty second windows.

In some embodiments, the pre-processing step 220 may also includeprocessing the motion and cardiac data each window and/or period toremove any noise or outlier. For example, for the cardiac data and/ormotion data, the step 220 may include applying a threshold or a filter(e.g., bandpass filter) to the cardiac data and/or motion data to removeany outlier data and setting the outlier data to a set value, forexample, the corresponding threshold. In some embodiments, for cardiacdata, the step 220 may also include amplitude normalization for eachnon-overlapping window of data, sphering to adjust for inter-deviceoffsets, and bandpass filtering and resampling with an anti-aliasfilter.

In some embodiments, the step 220 may also include determining pulseonset detection for each window. For example, the pulse onset detectionmay be determined using a gradient slope thresholding technique, zerocrossing of an envelope function, an autocorrelation estimation process,or a model-based fitting process.

By way of example, for PPG data, for each normalization second windowsegment of each PPG color channel, outlier rejection and amplitudenormalization may be performed. For example, the lower 5-percentile andupper 95-percentile of the PPG signal within each window segment may becalculated after subtracting the signal mean. The lower and upperpercentile may correspond to the thresholds and any signal valuesurpassing these two thresholds can be set to the correspondingthreshold (e.g., extreme value clipping). After which, each window maybe normalized by the maximum value in the segment. Next, each PPGchannel can be bandpass filtered to remove frequencies outside a normalrange (e.g., range of 0.2-10 Hz, using an FIR filter of order 41).Within each window, PPG pulse onset detection (e.g., using the slope sumfunction (SSF) approach) can be determined.

In some embodiments, the method 200 may include a step 230 ofdetermining a signal quality associated with each window of the cardiacdata and the motion data of each time period. In some embodiments, thesignal quality may be represented by one or more signal qualityindices(s) (SQI). The step 230 may include determining one or moresignal quality indices for the cardiac data and the motion data. Forexample, the signal quality may be determined by using a templatematching process.

In some embodiments, the signal quality of the cardiac data may beanalyzed using two different detectors (heart beat or peak/slope). Byway of example, the signal quality data index for the cardiac data mayrelate to a proportion of disagreements in the detectors.

For example, for the cardiac data, a signal quality index may bedetermined for each window using two different beat detectors. Onedetector may be highly sensitive and one may be highly specific. By wayof example, the signal quality index for the cardiac data may bedetermined using the Hjorth's purity quality metric. The signal qualityindex may be a percentage of beats that are agreed upon by two beatdetectors with different noise responses. By way of example, for PPGand/or ECG data, a signal quality index may be determined for eachchannel in each window.

In some embodiments, the motion data may include one or more signalquality indices related to energy. In some embodiments, a signal qualityindex for motion data (may be determined using the average value of themagnitude of the accelerometer for each window. By way of example, foraccelerometer data, the first signal quality index may be determinedusing the average value of the magnitude of the accelerometer data(ACC=√{square root over (x²+y²+z²)}) within each window. In someembodiments, an alternative or additional quality index for the motiondata may be determined using the standard deviation of the motion datawithin each window. By determining the SQI for the motion data, largemovements that can cause low quality data can be identified.

In some embodiments, the signal quality index for each of the cardiacand motion data may be combined (e.g., Boolean sense) or may remainseparate variables, or a combination thereof.

In some embodiments, the step 230 may also include determining a qualitychannel for each window of cardiac data based on one or more signalquality indices determined for each channel in the respective window.The quality channel may correspond to the channel with the highestsignal quality (e.g., having the high signal index or indices) and thedata (i.e., features) from this channel can be selected to represent thecorresponding window.

In some embodiments, the step 230 may include determining a qualitywindow for the motion data. The quality window may correspond to thewindow having the lowest amount of energy (e.g., is lowest accelerationenergy window). In some embodiments, the step 230 may includedetermining a quality window for each period using the SQI for thecardiac and motion data. The quality window for each period maycorrespond to the window with the highest signal quality and least ACC.

In some embodiments, the method 200 may optionally include a step 240 ofdetermining one or more subject information features (also referred toas “subject contextual information features”) for example, from thesubject information collected by the subject information device 140. Thesubject information features may relate to any relevant contextualinformation, such as medical history and/or demographics. For example,the medical history information and/or demographic information may beprocessed to determine or identify the history of certain medications(e.g., beta-blockers, calcium blockers, blood thinners), dosage ofmedications, device usage (e.g., pacemaker), history of certain medicalevents (e.g., stroke, myocardial infraction, etc.), age, gender, weight,among others, or a combination thereof. Such information can modify theinterpretation of features at various stages of processing. In someembodiments, the subject information features may be stored by thesystem 100 after an initial determination for the subject for processingthe set(s) of cardiac and motion data received for the patient.

In some embodiments, the method 200 may include a step 250 of extractingone or more cardiovascular features using the (raw and/or pre-processed)cardiac data, the (raw and/or pre-processed) motion data, one or moreSQI (e.g., cardiac and/or motion SQI), the subject information(features), among others, or any combination thereof. By way of example,one or more cardiovascular features may be based onbeat-to-beat-interval variations. For example, the one or morecardiovascular features may include but are not limited to one or moreentropy or cross-entropy related features, one or more standarddeviation features, among others, or any combination thereof. Thefeatures may be determined for each channel, all channels, betweencahllens (cross information) or a combination thereof.

By way of example, for PPG and/or ECG data, one or more cardiovascularfeatures for each window may be determined using the cardiac data forone channel, such as the quality channel determined for that window

By way of example, a first sample entropy feature for the cardiac datamay be determined for each window with the embedding dimensions m=1, and2. For example, a first standard deviation feature for the cardiac datamay be determined by determining a standard deviation of all channels ofall windows of the period using the raw cardiac data for the period oftime. A second standard deviation feature, which is a more robustversion of the standard deviation may also be determined for allchannels of all windows using the pre-processed cardiac data, because itdiscards the intervals outside the 0.05-0.95 percentile range.

For example, a third weighted standard deviation feature may bedetermined for the motion data. By way of example, for accelerometerdata, the inverse of the ACC waveform within each window may be used asthe weighing factor for when calculating the standard deviation.

In some embodiments, the method 200 may include a step 260 of applying atensor transform applying a tensor transformation to the cardiac dataand/or motion data for each window to transform the data totime-frequency space. The tensor transformation may include but is notlimited to wavelet transform, short-time Fourier transform (STFT), aGabor transform, a compressed matrix, among others, or a combinationthereof. In some embodiments, the tensor transform may be performed onthe cardiac data associated with the highest quality channel for eachwindow. The tensor transform may result in a tensor for each window. Insome embodiments, the tensor transformation may be applied to the rawcardiac data and/or motion data associated with each window and/orchannel.

In some embodiments, the tensor transform may be a uni-modal tensortransformation (such as wavelet transform of the cardiac data) or amulti-modal tensor transformation (such a cross-wavelet transform ofmotion and cardiac data). For example, the motion data may includerespiratory activity (which can depend on the location of the motionsensor) and the resulting cardio-respiratory cross-wavelet transform (orother transform) can represents the cross-frequency coupling. Suchinformation can be useful for arrhythmia detection as it can enable theclassifier (step 270) to distinguish between respiratory inducedvariability in rhythm versus arrhythmia related rhythm irregularity.

In some embodiments, the tensor transformation may be a wavelettransformation. For example, the size of the wavelet spectrum computedmay be sized 125×125 and the mother wavelet may be the “Morlet wavelet.”After which, the derived wavelet power spectrum may be further processedto remove the contribution of noise to the spectrum, for example, bynormalizing each window using the maximum wavelet power determined forthe period of cardiac data and/or motion data (i.e., all windows forthat period)(depending on whether the transformation is uni-modal ormulti-modal). By way of example, the statistical significance level ofthe wavelet power may be estimated, for example, using a Monte Carlomethod. Next, a large ensemble of surrogate data (N=100) may begenerated with the same first order autoregressive (R1) coefficients asthe input signals. For each surrogate data, a wavelet power for eachwindow may be calculated and thresholded using a maximum wavelet power(e.g., using the 95-percentile of the power as the threshold above whichthe observed signal power can be considered statistically significant(at 5% significance level)). The resulting wavelet power spectrum foreach window (the wavelet power above the threshold) corresponds to theresulting tensor corresponding to the window. Each tensor may correspondto a window of cardiac data or a window of cardiac and motion datacombined.

By way of example, FIG. 4 shows an example of a wavelet transformationbeing performed on a window 410 of cardiac data and a window 420 ofcardiac data. As shown in FIG. 4, the transformation results in tensors412 and 422 in which the cardiac data may be mapped into a largerrepresentation that emphasizes the differences in patterns betweenabnormal cardiac activity (i.e., atrial fibrillation) and other classes(i.e., normal cardiac activity (normal sinus rhythm) or noise),respectively.

In some embodiments, the method 200 may include a step 270 ofdetermining cardiac activity using one or more neural networks based oncardiac SQI (step 230), motion SQI (step 230), user information features(step 240), cardiovascular features (step 250), or any combinationthereof. For example, the step 270 may determine whether the subject hadan occurrence of abnormal cardiac activity (e.g., atrial fibrillation)at a time point within the time period and the time point associatedwith each occurrence of abnormal cardiac activity.

In some embodiments, the step 270 may include processing the tensor foreach window (step 260) using one or more neural networks to determinecardiac activity associated with each window. The one or more neuralnetworks may include neural networks that differ from each other in atleast one of architecture and/or functionality (e.g., convolutionalneural networks, recurrent neural networks), feature input and/or type(e.g., cardiac, motion, user information features), among others, or acombination thereof. In some embodiments, the cardiac activity may bedetermined using an attention network (mechanism) that is connected inseries to other neural network(s).

In some embodiments, the one more neural networks may include aconvolutional neural network that extracts deep learning features fromthe tensor for each window (e.g., as a feature vector) and abidirectional recurrent neural network (RNN) that maps the deep learningfeatures, cardiac and/or motion SQI indices, and user information, to aclass probability. In some embodiments, the one or more neural networksmay also include an attention network. The attention network maydetermine the optimal sections of data to analyze to provide a weightfor each window of a time period. The attention network may use theweighted combinations of windows to determine a classification of thesubject. The classification using the attention network may result in adiagnosis of abnormal cardiac activity, type of abnormal cardiacactivity, and/or burden of the abnormal cardiac activity (i.e.,percentage of time that a patient exhibits abnormal cardiac activity(total time in abnormal cardiac activity (e.g., AF) divided by theperiod of time)). Using the weights from the attention model, the windowin which the abnormal cardiac activity (e.g., arrhythmia) occurs may beidentified. For example, the window with the highest weight may beconsidered the window in which the abnormal cardiac activity occurred.This can be provided to the clinician in a report.

The one or more neural networks may be a trained multi-class classifier.In some embodiments, the deep convolutional network may be trained byprocessing on a set of tensors to learn one or more deep learningfeatures associated with abnormal and normal cardiac activity.

In some embodiments, the resulting networks (CNN, RNN and attentionnetwork) may then trained together (end-to-end), for example, trainingthe attention network and RNN using the learning features extracted bythe CNN, so as to optimize the classifier.

In some embodiments, the method 200 may include a step 280 of compilingand outputting the results of the determination of cardiac activity forone or more periods of time. For example, the results (step 270) may betransmitted (e.g., to the clinician), printed, displayed, stored, amongothers, or any combination thereof. In some embodiments, the results maybe outputted as a report. In some embodiments, the report may includethe results of the determination for one or more periods of time (step270). For example, the results of the determination of cardiac activity(step 270) may include a diagnosis of abnormal or normal cardiacactivity, type of abnormal cardiac activity, and/or a burden of cardiacactivity; one or more windows including abnormal cardiac activity withthe abnormal cardiac activity identified; among others, or combinationthereof.

In some embodiments, the report may compile the results of the analysisand classification for more than one period of time. For example, aspreviously noted, the system 100 may receive a set of cardiac and motiondata for a length of time (e.g., 1 hour) and may separate that into aplurality of periods of time (e.g., six periods of time). In thisexample, the system 100 may repeat steps 220-270 for each period of timeand compile the results for the length of time in the output (e.g.,report) (step 280). In this example, if abnormal cardiac activity isdetermined in more than one period (e.g., at least one window in oneperiod is determined to include abnormal cardiac activity), the burdenmay be determined using all periods of time.

FIG. 3 shows an example of a method 300 of determining cardiac activityusing the tensor 310 for each window (from step 260), based on SQI forcardiac data (from step 230), SQI for motion data (step 230), one ormore cardiovascular features (step 250), and/or user informationfeatures (from step 240) for each period of time according to someembodiments.

In some embodiments, the method 300 may include a step 320 of encodingeach tensor 310 for each window of the period (from step 260) into oneor more deep learning features (e.g., feature vectors) associated withcardiac activity (e.g., abnormal and normal cardiac activity) using thedeep convolutional network (CNN) according to embodiments.

By way of example, the CNN can extract deep learning features from eachtensor, which is in a 2 dimensional space, and project it to a smaller 1dimensional feature vector. In some embodiments, the CNN may include oneor more convolutional and max-pool layers. In some embodiments, the CNNmay include ten or more convolutional and fully connected layers. Nodesmay be automatically pruned, and optimized to act as filters to createfeature vectors (i.e., deep learning features) from the input tensor ineach window.

FIG. 5 shows an example 500 of a method of encoding of regions of eachtensor for each window into one or more deep learning features (e.g.,feature vectors) using the deep convolutional network according to someembodiments.

In some embodiments, the method 300 may further include a step 330 ofclassifying each window of cardiac data and/or motion data into one moreclasses of each period of time using the deep learning features(vectors) representing each window (step 320); the signal qualityindices 342 and/or 346 for the respective window (from step 230), one ormore subject information features 346 (step 240), and/or one or more ofthe cardiovascular features 348 (step 250). In some embodiments, theclassifying 330 of each window of the period may be performed by abidirectional recurrent neural network. In some embodiments, theclassifying 330 may determine a probability that each window belongs toa class of one or more classes. The classes may include but are notlimited to abnormal cardiac activity (e.g., type, presence, etc.),normal cardiac activity, noise, among others, or a combination thereof.

In some embodiments, the bidirectional recurrent neural network mayinclude stacking multiple layers to predict the probability (or weight)of each deep learning feature for each window belonging to a class, withthe output sequence of one layer forming the input sequence for thenext. FIG. 6 shows an example of a bidirectional recurrent neuralnetwork that encodes the CNN feature vector, in addition to thecardiovascular features, SQI, and/or subject information features, toaccount for temporal characteristics of the signal. As shown in thisfigure, the input, X, which represents the CNN feature vector, inaddition to the cardiovascular features, SQI, and/or subject informationfeatures, for each window (1, 2, . . . T), can be fed into abidirectional RNN.

The forward and backward passes can allow the neural network to expressan opinion (i.e., a weight) regarding cardiac activity (e.g., abnormalcardiac activity) at each time-step (each window), by analyzing the setof inputs for the current window (e.g., i), and (1) the previous window(e.g., i−1) and (2) the following window (i+1). For example, for theforward pass, the input (X) for each window (i)(e.g., (1, 2, . . . T))can be inputted into the forward layer, which are used to update thehidden units of the RNN. The backward pass may include updating thehidden units and outputting a class probability vector (Y_(i)) for eachwindow (i=1, 2, . . . , T) in the reverse direction for further analysis(e.g., using the outputs from the previous window and following window).After the forward pass of all inputs (X_(i), i=1, 2, . . . , T) for theperiod and update of the hidden units, the backward pass can ensure thatthe hidden units include information from both proceeding and succeedingwindows (also referred to as the “smoothed hidden units”). The smoothedhidden units can then be used to update the corresponding outputs(Y_(i), i=1, 2, . . . , T), which can then be inputted into theattention network.

In some embodiments, the method 300 may further include a step 350 ofdetermining cardiac activity associated with each window of each periodof time based on the classification from step 330.

In some embodiments, the step 350 may include inputting the output(Y_(i), i=1, 2, . . . , T) from step 330 (i.e., from RNN) into anattention network to determine a diagnosis related to abnormal cardiacactivity (e.g., incidence of atrial fibrillation) or burden of abnormalcardiac activity. In some embodiments, the attention network may providea score as an output for each window. The score may correspond to aweighted combination of the weights (or probabilities) determined foreach window (step 330). In some embodiments, the score may be indicativeof the class (e.g., abnormal, normal, noisy, other, etc.).

In some embodiments, the attention network can determine the optimalsections of data to analyze, or how to relatively weigh combinations ofwindows to provide a final diagnosis or burden. It can also identify thetime(s) at which the cardiac activity occurred. This way, the clinicianmay review the results of the systems and devices of the disclosure.

For example, FIG. 7 shows an example of an attention network accordingto embodiments. In this example, the final output, Y_(attention),corresponding to the classification of the cardiac activity of a subjectfor a period of time, can be formed by taking all of the Y(RNN outputs)for each window (i), Y_(i), and weighting them by the degree ofimportance each window of cardiac data should have in the period oftime. The outputs (Y_(i), i=1, 2, . . . , T) from the RNN (step 350) maythen be weighted by the corresponding attention model weights (w_(i)),to produce the final outputs for each period. The final outputs(Y_(attention)) for each window of each period corresponds toY_(attention)=Σ_(i=1) ^(T)w_(i)Y_(i). The weight for each window can bebased on criteria, including but not limited to: cardiac SQI, motionSQI, tensor(s), probability (output from step 330), health information,among others, or a combination thereof. This weighting, or “attention”can determined by optimization of the weights over the entire trainingset, using back propagation. The score may correspond to a sum of eachweight determined for the window.

In some embodiments, the window having the highest weight (or score) ineach period may be considered the window time having an abnormality.

By way of example, for a multi-class classification of subject data withclasses: normal cardiac activity, Atrial Fibrillation, and noisy. If theperiod includes 2 windows and Y_(i)=[0.1 0.6 0.3] with w₁=0.3, andY₂=[0.1 0.3 0.6] with w₂=0.7, the final output from the attention modelor score (Y_(attention)) can be calculated as follows: 0.3×[0.1 0.60.3]+0.7×[0.1 0.3 0.6]. The score (Y_(attention)) may then equal [0.10000.39 0.51]. Based on that score, the system may determine that period isassociated most likely with the class “noisy” (corresponding toprobability of 0.51).

Computer System

One or more of the devices and/or systems of the system 100 may beand/or include a computer system and/or device. FIG. 8 is a blockdiagram showing an example of a computer system 800. The modules of thecomputer system 800 may be included in at least some of the systemsand/or modules, as well as other devices and/or systems of the system100.

The system for carrying out the embodiments of the methods disclosedherein is not limited to the systems shown in FIGS. 1 and 8. Othersystems may also be used. It is also to be understood that the system800 may omit any of the modules illustrated and/or may includeadditional modules not shown.

The system 800 shown in FIG. 8 may include any number of modules thatcommunicate with each other through electrical or data connections (notshown). In some embodiments, the modules may be connected via anynetwork (e.g., wired network, wireless network, or a combinationthereof).

The system 800 may be a computing system, such as a workstation,computer, or the like. The system 800 may include one or more processors812. The processor(s) 812 may include one or more processing units,which may be any known processor or a microprocessor. For example, theprocessor(s) may include any known central processing unit (CPU),graphical processing unit (GPU) (e.g., capable of efficient arithmeticon large matrices encountered in deep learning models), among others, orany combination thereof. The processor(s) 812 may be coupled directly orindirectly to one or more computer—readable storage media (e.g., memory)814. The memory 814 may include random access memory (RAM), read onlymemory (ROM), disk drive, tape drive, etc., or a combinations thereof.The memory 814 may be configured to store programs and data, includingdata structures. In some embodiments, the memory 814 may also include aframe buffer for storing data arrays.

In some embodiments, another computer system may assume the dataanalysis or other functions of the processor(s) 812. In response tocommands received from an input device, the programs or data stored inthe memory 814 may be archived in long term storage or may be furtherprocessed by the processor and presented on a display.

In some embodiments, the system 800 may include a communicationinterface 816 configured to conduct receiving and transmitting of databetween other modules on the system and/or network. The communicationinterface 816 may be a wired and/or wireless interface, a switchedcircuit wireless interface, a network of data processing devices, suchas LAN, WAN, the internet, or combination thereof. The communicationinterface may be configured to execute various communication protocols,such as Bluetooth, wireless, and Ethernet, in order to establish andmaintain communication with at least another module on the network.

In some embodiments, the system 810 may include an input/outputinterface 818 configured for receiving information from one or moreinput devices 820 (e.g., a keyboard, a mouse, and the like) and/orconveying information to one or more output devices 820 (e.g., aprinter, a CD writer, a DVD writer, portable flash memory, etc.). Insome embodiments, the one or more input devices 820 may be configured tocontrol, for example, the generation of the management plan and/orprompt, the display of the management plan and/or prompt on a display,the printing of the management plan and/or prompt by a printerinterface, the transmission of a management plan and/or prompt, amongother things.

In some embodiments, the disclosed methods (e.g., FIGS. 2 and 3) may beimplemented using software applications that are stored in a memory andexecuted by the one or more processors (e.g., CPU and/or GPU) providedon the system 100. In some embodiments, the disclosed methods may beimplemented using software applications that are stored in memories andexecuted by the one or more processors distributed across the system.

As such, any of the systems and/or modules of the system 100 may be ageneral purpose computer system, such as system 800, that becomes aspecific purpose computer system when executing the routines and methodsof the disclosure. The systems and/or modules of the system 100 may alsoinclude an operating system and micro instruction code. The variousprocesses and functions described herein may either be part of the microinstruction code or part of the application program or routine (orcombination thereof) that is executed via the operating system.

If written in a programming language conforming to a recognizedstandard, sequences of instructions designed to implement the methodsmay be compiled for execution on a variety of hardware systems and forinterface to a variety of operating systems. In addition, embodimentsare not described with reference to any particular programming language.It will be appreciated that a variety of programming languages may beused to implement embodiments of the disclosure. An example of hardwarefor performing the described functions is shown in FIGS. 1 and 8. It isto be further understood that, because some of the constituent systemcomponents and method steps depicted in the accompanying figures can beimplemented in software, the actual connections between the systemscomponents (or the process steps) may differ depending upon the mannerin which the disclosure is programmed. Given the teachings of thedisclosure provided herein, one of ordinary skill in the related artwill be able to contemplate these and similar implementations orconfigurations of the disclosure.

While the disclosure has been described in detail with reference toexemplary embodiments, those skilled in the art will appreciate thatvarious modifications and substitutions may be made thereto withoutdeparting from the spirit and scope of the disclosure as set forth inthe appended claims. For example, elements and/or features of differentexemplary embodiments may be combined with each other and/or substitutedfor each other within the scope of this disclosure and appended claims.

1. A computer-implemented method for using machine learning to determineabnormal cardiac activity of a subject, the method comprising: receivingone or more periods of time of cardiac data and motion data for asubject, each period of time including more than one window of thecardiac data and the motion data; determining one or more signal qualityindices for each window of the cardiac data and the motion data of theone or more periods of time; extracting one or more cardiovascularfeatures for each period of time using at least the cardiac data, themotion data, and the one or more signal quality indices for the cardiacdata and the motion data; applying a tensor transform to the cardiacdata and/or the motion data to generate a tensor for each window of theone or more periods of time; applying a trained deep learningarchitecture to each tensor of the one or more periods of time toclassify each window and/or each period into one or more classes usingat least the one or more signal quality indices for the cardiac data andthe motion data and cardiovascular features, the deep learningarchitecture including a convolutional neural network, a bidirectionalrecurrent neural network, and an attention network, the one or moreclasses including abnormal cardiac activity and normal cardiac activity;and generating a report including a classification of cardiac activityof the subject for the one or more periods based on the one or moreclasses.
 2. The method according to claim 1, further comprising:receiving subject contextual information for the subject, the subjectcontextual information including medical history and demographicinformation; wherein the extracting uses one or more subject informationfeatures related to the subject contextual information to extract one ormore cardiovascular features for each period of time, and the traineddeep learning architecture uses the one or more subject informationfeatures to classify the cardiac activity for each window of the period.3. The method according to claim 1, wherein the tensor transform isapplied to the cardiac data and the motion data for each window.
 4. Themethod according to claim 3, further comprising: determining a qualitychannel for each window based on the one or more signal quality indicesfor the cardiac data and the motion data, the quality channelcorresponding to a channel in each window having the one more qualityindices that is higher than remaining channels in each channel.
 5. Themethod according to claim 1, wherein the applying the deep learningarchitecture includes: encoding each tensor for each window of the oneor more periods using the deep convolutional network into one or moredeep learning features associated with cardiac activity; applying thebidirectional recurrent network to determine a probability that eachwindow of the one or more periods belongs to a class of the one or moreclasses, the bidirectional recurrent network using the one or more deeplearning features, the one more signal quality indices for the cardiacdata and/or motion data, and/or one or more cardiovascular features toclassify each window of the one or more periods; and determining theclassification of cardiac activity for each window of the one or moreperiods and/or each period by applying the attention network to theprobability for each window of the one or more periods.
 6. The methodaccording to claim 5, wherein the attention network determines a scorefor each window and/or each period, the score representing theclassification of cardiac activity.
 7. The method according to any ofclaim 6, wherein when the classification of cardiac activity includesabnormal cardiac activity, a window of each period having a highestscore represents the window including the abnormal cardiac activity. 8.A non-transitory computer-readable storage medium storing instructionsfor using machine learning to determine abnormal cardiac activity of asubject, the instructions comprising receiving one or more periods oftime of cardiac data and motion data for a subject, each period of timeincluding more than one window of the cardiac data and the motion data;determining one or more signal quality indices for each window of thecardiac data and the motion data of the one or more periods of time;extracting one or more cardiovascular features for each period of timeusing at least the cardiac data, the motion data, and the one or moresignal quality indices for the cardiac data and the motion data;applying a tensor transform to the cardiac data and/or the motion datato generate a tensor for each window of the one or more periods of time;applying a trained deep learning architecture to each tensor of the oneor more periods of time to classify each window and/or each period intoone or more classes using at least the one or more signal qualityindices for the cardiac data and the motion data and cardiovascularfeatures, the deep learning architecture including a convolutionalneural network, a bidirectional recurrent neural network, and anattention network, the one or more classes including abnormal cardiacactivity and normal cardiac activity; and generating a report includinga classification of cardiac activity of the subject for the one or moreperiods based on the one or more classes.
 9. The medium according toclaim 8, the instructions further comprising: receiving subjectcontextual information for the subject, the subject contextualinformation including medical history and demographic information;wherein the extracting uses one or more subject information featuresrelated to the subject contextual information to extract one or morecardiovascular features for each period of time, and the trained deeplearning architecture uses the one or more subject information featuresto classify the cardiac activity for each window of the period.
 10. Themedium according to claim 8, wherein the tensor transform is applied tothe cardiac data and the motion data for each window.
 11. The mediumaccording to claim 10, further comprising: determining a quality channelfor each window based on the one or more signal quality indices for thecardiac data and the motion data, the quality channel corresponding to achannel in each window having the one more quality indices that ishigher than remaining channels in each channel.
 12. The medium accordingto claim 8, wherein the applying the deep learning architectureincludes: encoding each tensor for each window of the one or moreperiods using the deep convolutional network into one or more deeplearning features associated with cardiac activity; applying thebidirectional recurrent network to determine a probability that eachwindow of the one or more periods belongs to a class of the one or moreclasses, the bidirectional recurrent network using the one or more deeplearning features, the one more signal quality indices for the cardiacdata and/or motion data, and/or one or more cardiovascular features toclassify each window of the one or more periods; and determining theclassification of cardiac activity for each window of the one or moreperiods and/or each period by applying the attention network to theprobability for each window of the one or more periods.
 13. The mediumaccording to claim 12, wherein: the attention network determines a scorefor each window and/or each period, the score representing theclassification of cardiac activity; and when the classification ofcardiac activity includes abnormal cardiac activity, a window of eachperiod having a highest score represents the window including theabnormal cardiac activity.
 14. A system for using machine learning todetermine abnormal cardiac activity of a subject, comprising: a memory;and one or more processors, wherein the one or more processors isconfigured to cause: receiving one or more periods of time of cardiacdata and motion data for a subject, each period of time including morethan one window of the cardiac data and the motion data; determining oneor more signal quality indices for each window of the cardiac data andthe motion data of the one or more periods of time; extracting one ormore cardiovascular features for each period of time using at least thecardiac data, the motion data, and the one or more signal qualityindices for the cardiac data and the motion data; applying a tensortransform to the cardiac data and/or the motion data to generate atensor for each window of the one or more periods of time; applying atrained deep learning architecture to each tensor of the one or moreperiods of time to classify each window and/or each period into one ormore classes using at least the one or more signal quality indices forthe cardiac data and the motion data and cardiovascular features, thedeep learning architecture including a convolutional neural network, abidirectional recurrent neural network, and an attention network, theone or more classes including abnormal cardiac activity and normalcardiac activity; and generating a report including a classification ofcardiac activity of the subject for the one or more periods based on theone or more classes.
 15. The system according to claim 14, wherein theprocessor is further configured to cause: receiving subject contextualinformation for the subject, the subject contextual informationincluding medical history and demographic information; wherein theextracting uses one or more subject information features related to thesubject contextual information to extract one or more cardiovascularfeatures for each period of time, and the trained deep learningarchitecture uses the one or more subject information features toclassify the cardiac activity for each window of the period.
 16. Thesystem according to claim 14, wherein the tensor transform is applied tothe cardiac data and the motion data for each window.
 17. The systemaccording to claim 16, further comprising: determining a quality channelfor each window based on the one or more signal quality indices for thecardiac data and the motion data, the quality channel corresponding to achannel in each window having the one more quality indices that ishigher than remaining channels in each channel.
 18. The system accordingto claim 14, wherein the applying the deep learning architectureincludes: encoding each tensor for each window of the one or moreperiods using the deep convolutional network into one or more deeplearning features associated with cardiac activity; applying thebidirectional recurrent network to determine a probability that eachwindow of the one or more periods belongs to a class of the one or moreclasses, the bidirectional recurrent network using the one or more deeplearning features, the one more signal quality indices for the cardiacdata and/or motion data, and/or one or more cardiovascular features toclassify each window of the one or more periods; and determining theclassification of cardiac activity for each window of the one or moreperiods and/or each period by applying the attention network to theprobability for each window of the one or more periods.
 19. The systemaccording to claim 18, wherein the attention network determines a scorefor each window and/or each period, the score representing theclassification of cardiac activity.
 20. The system according to claim19, wherein when the classification of cardiac activity includesabnormal cardiac activity, a window of each period having a highestscore represents the window including the abnormal cardiac activity.